ci: Effective cache sharing between PRs #202
Merged
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
As mentioned in #186 (comment), our caching is currently sub-optimal and only really works for subsequent runs on the same PR.
Ideally, we'd like to share caches between PRs such that not every PR has to first build a cache. The way GHA caches work is that they can only be accessed if they were created by the same PR or by the target branch. I've thus added a workflow to (re-)create the pre-commit cache whenever we merge into the
developbranch. The linting workflow (running on PRs) will just restore the cache (and not attempt to upload a new one).Caching the test_data needs to be done in the workflow that creates the cache (e.g., for the "PyFV3 translate tests" we need to modify the workflow(s) in the PyFV3 repo).
We could cache the pip cache for "NDSL unit tests". However, since we (mostly) aren't pinning python package dependencies to specific version numbers (in
setup.pyandpyproject.toml), this cache would get out of date soon. And since the runtime of NDSL unit tests are governed by the test runtime, I don't see a need for that right now. Once we pin package dependencies with a lock file, we can re-evaluate.The changes in the pull request template are unrelated. They just stop my editor from complaining about invalid markdown syntax in that file. The changes are due to the following rules
# Header**bold font**as headersHow Has This Been Tested?
Tested the new workflow on my fork of NDSL.
Checklist